<!DOCTYPE html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<link rel="stylesheet" href="stylesheets/main.css" type="text/css" />
<link rel="icon" type="image/png" href="https://randoop.github.io/randoop/images/randoop-logo.png" />
<title>Randoop Developer's Manual</title>
</head>
<body>

<img src="https://randoop.github.io/randoop/images/randoop-logo.png" alt="Randoop logo" style="float:right" />

<h1>Randoop Developer's Manual</h1> <!-- omit from toc -->

<!-- start toc.  do not edit; run html-update-toc instead -->
<ul>
  <li><a href="#starting">Getting started</a>
    <ul>
      <li><a href="#prerequisites">Prerequisites</a></li>
    </ul></li>
  <li><a href="#compiling">Building Randoop</a>
    <ul>
      <li><a href="#publish-only-tasks">Release-only tasks</a></li>
      <li><a href="#gradlew">The Gradle wrapper</a></li>
      <li><a href="#buildgradle">The Randoop build script</a></li>
    </ul></li>
  <li><a href="#running-randoop">Running Randoop</a>
    <ul>
      <li><a href="#classpath">Randoop classpath</a></li>
    </ul></li>
  <li><a href="#testing-randoop">Testing Randoop</a>
    <ul>
      <li><a href="#coverage-tests">Checking Randoop code coverage</a></li>
      <li><a href="#travis-tests">Running tests under Travis-CI, and reproducing them outside Travis</a></li>
      <li><a href="#addtests">Adding tests</a>
        <ul>
          <li><a href="#unit-tests">Unit tests</a></li>
          <li><a href="#system-tests">System tests</a></li>
        </ul></li>
    </ul></li>
  <li><a href="#editing-randoop">Editing Randoop</a>
    <ul>
      <li><a href="#usinganide">Using an IDE</a></li>
      <li><a href="#documentingchanges">Documenting changes</a></li>
      <li><a href="#codeformatting">Formatting Randoop source code</a></li>
      <li><a href="#maintaininglibraries">Updating libraries</a></li>
      <li><a href="#modmanual">Modifying the manual</a></li>
      <li><a href="#github-site">Modifying the website</a></li>
    </ul></li>
  <li><a href="#making_new_dist">Releasing a new version of Randoop</a></li>
  <li><a href="#randoop_internals">Randoop internals</a>
    <ul>
      <li><a href="#unit_tests">Unit test concepts</a></li>
      <li><a href="#sequence">Sequences</a>
        <ul>
          <li><a href="#creating_sequence">Creating sequences</a></li>
        </ul></li>
      <li><a href="#executing_sequence">Executable sequences</a>
        <ul>
          <li><a href="#executing-a-sequence">Executing a sequence</a></li>
          <li><a href="#miscellaneous-notes">Miscellaneous notes</a></li>
        </ul></li>
      <li><a href="#inout_sequence">Writing/reading sequences to file</a>
        <ul>
          <li><a href="#writing_sequence_as_junit">Writing a sequence as a JUnit test</a></li>
        </ul></li>
      <li><a href="#checks">Checks</a>
        <ul>
          <li><a href="#distinction">Statements vs. checks</a></li>
          <li><a href="#checks_">Executing Checks</a></li>
        </ul></li>
      <li><a href="#code-entry">Main entry points</a></li>
      <li><a href="#code-generator">Test generator classes</a></li>
    </ul></li>
</ul>
<!-- end toc -->

<p>
(Also see the <a href="index.html">Randoop Manual</a>.)
</p>


<hr />



<h1 id="starting">Getting started</h1>

<p>Clone the repository, using either of these commands:</p>

<pre>
  git clone git@github.com:randoop/randoop.git
  git clone https://github.com/randoop/randoop.git
</pre>

<p>
  When working on a particular feature, create a new
  branch and work there rather
  than in the <code>master</code> branch.
</p>


<h2 id="prerequisites">Prerequisites</h2>

<p>
  To install prerequisites, execute (as superuser) the RUN commands in files
  named <code>scripts/Dockerfile-<em>OPERATINGSYSTEM</em>-jdkany</code>.
  For example, to install prerequisites on Ubuntu, run
</p>
<!-- Accurate as of July 2017 -->
  <pre>
    sudo apt-get -qqy install git gradle default-jdk
    sudo apt-get -qqy install perl python-pip
    sudo pip install html5validator
  </pre>

<p>
  If you do not have superuser permissions, ask your system administrator
  to install the software.  You can run the last command yourself, by changing it to
  <code>pip install --user html5validator</code>.
</p>

<!--
Unfortunately, gradle seems to require that html5validator is on the path, even if
you will never run a task that calls html5validator.
-->


<h1 id="compiling">Building Randoop</h1>

<p>
  In Randoop's top-level directory, run
</p>
<pre>  ./gradlew build manual</pre>
<p>
  This command compiles Randoop, compiles the Java agents, runs all tests,
  builds the jar files, and updates the manual from the source Javadoc.
  Generated files are
  placed in subdirectories named <code>build</code> in the parent directory and in
  each of the agent subdirectories.
  Jar files are placed in directory <code>build/libs</code>.
</p>

<p>
  <b>Windows users:</b> you do not need to include the "<code>./</code>" when running Gradle.
  When our instructions say to run <code>./gradlew build</code>, you will run <code>gradlew build</code> instead.
</p>

<p>
  You can also break this process down by running separate tasks.
  For the full list of tasks supported by the Randoop build script, type
  <code>./gradlew tasks</code>.
  Particular tasks that may be useful during development include
</p>
<table style="border:0;">
  <tr>
    <td><code>./gradlew jar</code></td>
    <td>to build the main Randoop jar file (but not agent or fat jar files)</td>
  </tr>
  <tr>
    <td><code>./gradlew assemble</code></td>
    <td>to compile and build Randoop and fat jar files</td>
  </tr>
  <tr>
    <td><code>./gradlew shadowJar</code></td>
    <td>to build the "user" jar including all library dependencies
  </tr>
  <tr>
    <td><code>./gradlew check</code></td>
    <td>to run all of the tests</td>
  </tr>
  <tr>
    <td><code>./gradlew testReport</code></td>
    <td>to generate HTML reports from tests
      (in <code>build/reports/allTests</code>)</td>
  </tr>
  <tr>
    <td><code>./gradlew clean</code></td>
    <td>to remove the <code>build</code> subdirectory</td>
  </tr>
  <tr>
    <td><code>./gradlew compileJava</code></td>
    <td>to compile the <code>main</code> source set</td>
  </tr>
  <tr>
    <td><code>./gradlew compileTestJava</code></td>
    <td>to compile the <code>test</code> source set</td>
  </tr>
  <tr>
    <td><code>./gradlew compileCoveredTestJava</code></td>
    <td>to compile the <code>coveredTest</code> source set</td>
  </tr>
  <tr>
    <td><code>./gradlew manual</code></td>
    <td>to update the manual from the source Javadoc</td>
  </tr>
</table>
<p>
  Compiling uses the concept of
  <a href="https://docs.gradle.org/current/userguide/java_plugin.html"><i>source sets</i></a>.
  A source set is a grouping of source files that can have its own set of
  dependencies and tasks.
  For instance, in the Randoop <code>build.gradle</code> there is the
  <code>coveredTest</code> and <code>replacecallTest</code> source sets that
  are separate from the <code>test</code> sourceset because the test classes
  must run using the <code>covered-class</code> and <code>replacecall</code> Java
  agents.
  The Gradle Java plugin normally has two source sets named <code>main</code> and <code>test</code>,
  and the Randoop build script defines others to better structure the build process.
</p>

<p>
Gradle allows you to run multiple tasks.  For instance, for a clean build,
    type <code>./gradlew clean build</code>.
</p>


<h2 id="publish-only-tasks">Release-only tasks</h2>

<p>
The tasks <code>cleanSite</code>, <code>publishSite</code> and <code>buildRelease</code>
should not be run except while making a release, and so should never be run
within a branch other than master.
These tasks will delete and/or replace the GitHub pages site located in the <code>docs</code> directory of the repository.
If you happen to run these commands, you should use git to revert to the previous version.
For instance, the command
</p>

<pre><code>
  git checkout -- docs/
</code></pre>

<p>
can be used to restore the files before they have been committed.
</p>



<h2 id="gradlew">The Gradle wrapper</h2>

<p>
Randoop is built with Gradle.
The shell script named <code>./gradlew</code> and the Windows batch script
<code>gradlew.bat</code> run the Gradle wrapper for the project, which is the
recommended way to run Gradle commands.
Using the wrapper means you don't have to install Gradle yourself, instead
the wrapper will download Gradle the first time you run a build command.
</p>

<p>
To learn more about Gradle beyond the basic commands
given here, see the Gradle
<a href="https://docs.gradle.org/current/userguide/command_line_interface.html">command-line tutorial</a>.
</p>


<h2 id="buildgradle">The Randoop build script</h2>

<p>
  The Gradle build tasks for Randoop are defined in the build script <code>build.gradle</code>,
  along with the build scripts for the java agents in the corresponding subproject directories.
  This is the file you would modify to change the Randoop build.
  Note that the script uses plugins that each define tasks that may not be explicit
  within the script, though it may be possible to configure them differently by
  following the instructions at the plugin sites.
</p>

<p>
  If you add or change a build task in a way that affects how Randoop is built,
  you should also update this document, which appears in the gh-pages branch.
  When you push your changes to the master branch on GitHub, you should also
  push to the gh-pages branch.
</p>


<h1 id="running-randoop">Running Randoop</h1>

<h2 id="classpath">Randoop classpath</h2>

<p>
  When running Randoop, you should use the file
  <code>randoop-all-<em>X.Y.Z</em>.jar</code>
  that includes all dependencies.
  For more detail, see the <a href="index.html">Randoop Manual</a>.
</p>

<p>
The jar files for Java agents are located in
the <code>build/libs</code> subdirectory of both the Randoop project and the
agent project directories.
So, for instance, the following will work to run the <code>covered-class</code> agent:
</p>
<pre><code>-javaagent:&lt;randoop-path&gt;/build/libs/covered-class-4.1.0.jar</code></pre>



<h1 id="testing-randoop">Testing Randoop</h1>

<p>
To run the Randoop tests:
</p>
<pre><code>./gradlew check</code></pre>

<p>
You can run a subset of the tests by using the <code>--tests</code>
option on the command line.
For instance,
</p>
<pre><code>    ./gradlew test --tests="*GenericTypesTest"
    # You can include "run" in the test name, but still use a leading "*"
    ./gradlew systemtest --tests="*Collection*"
</code></pre>

<p>
The system tests write logs to the directory
<code>build/working-directories/<em>test-name</em></code>.
You can create additional logs (as produced by
<a href="index.html#option:selection-log"><code>--selection-log</code></a> or
<a href="index.html#option:operation-history-log"><code>--operation-history-log</code></a>)
by setting the system properties
<code>randoop.selection.log</code> or
<code>randoop.operation.history.log</code> on the Gradle command line.
For instance:
</p>

<pre><code>  ./gradlew systemtest --tests="*Collection*" \
    -Drandoop.selection.log=selection-log.txt -Drandoop.operation.history.log=operation-log.txt
</code></pre>
<p>
  will create the files
</p>
<pre><code>  build/working-directories/<em>test-name</em>/randoop-log.txt
  build/selection-log.txt
  build/operation-log.txt
</code></pre>
<p>
Note that this works for system tests only, not for unit tests or other types of
tests.
Further note that selection logging can slow down the system tests so much that
they fail.
Finally, it should be noted that even though the default replacements attempt to suppress calls to
methods that use a graphical windowing system, not all such calls are captured.
Currently, the subtest {@code runDirectSwingTest} will fail with {@code java.awt.AWTError:
Local GraphicsEnvironment must not be null} if the system tests are run in a headless environment.
</p>

<p>
If Randoop generates an unexpected test (for example, Randoop's test suite outputs
<code>java.lang.AssertionError: Test suite should have no error tests, but has 1:</code>),
you can run the test with commands like these:

<pre>
javac -cp .:junit-4.11.jar -sourcepath .:.../randoop/src/testinput/java/ ErrorTest.java
java -cp .:junit-4.11.jar:.../randoop/build/classes/java/testInput/ org.junit.runner.JUnitCore ErrorTest
</pre>

You can minimize the test with a command like this:
<pre>
java -ea -cp randoop-all-4.1.0.jar randoop.main.Main minimize \
  --suitepath=ErrorTest0.java --suiteclasspath=.:junit-4.11.jar:.../randoop/classes/java/testInput/
</pre>


<h2 id="coverage-tests">Checking Randoop code coverage</h2>

<p>
  Before making a release, and to evaluate the effect of changes to Randoop, you should compute coverage of Randoop-generated tests for 4 codebases.
</p>

<ul>
  <li>Randoop's test suite.  It produces
reports in <code>build/reports/jacoco/test/html</code> and its subdirectories.
A summary of the results is in the file <code>build/reports/jacoco/test/jacocoTestReport.xml</code>.
The overall coverage can be viewed by running the perl script: <code>scripts/show-coverage.pl</code>.
This script will accept an optional argument of an alternative file location.
Invoke the script with <code>-help</code> for a full list of options.
If you are planning to update the Randoop coverage data spreadsheet <code>Randoop Coverage.xlsm</code>
(location TBD) you will need to run the perl script: <code>scripts/prepare-coverage.pl</code>.
This tool reads the jacoco coverage data and converts it into into a csv file named
"report-&lt;input file modifcation date&gt;.csv" in the current directory.
This script will accept an optional argument of an alternative file location.
Invoke the script with <code>-help</code> for a full list of options.
  </li>
  <li><a href="https://github.com/rjust/defects4j">Defects4j</a>: See the file <a href="https://github.com/rjust/defects4j/blob/master/framework/test/README.md">README.md</a> for details.</li>
  <li><a href="https://gitlab.cs.washington.edu/randoop/pascali-coverage">Pascali</a>: See the file <a href="https://gitlab.cs.washington.edu/randoop/pascali-coverage/blob/master/README.md">README.md</a> for details.</li>
  <li><a href="https://gitlab.cs.washington.edu/randoop/toradocu-coverage">Toradocu</a>: See the file <a href="https://gitlab.cs.washington.edu/randoop/toradocu-coverage/blob/master/README.md">README.md</a> for details.</li>
</ul>


<h2 id="travis-tests">Running tests under Travis-CI, and reproducing them outside Travis</h2>

<p>
<a href="https://travis-ci.org/randoop/randoop">Travis-CI runs the Randoop tests</a>.
The Travis tests are
equivalent to <code>./gradlew clean build</code>, and they
currently pass on the master branch if this icon is green:
<img src="https://api.travis-ci.org/randoop/randoop.svg?branch=master" alt="Randoop Travis status icon"/>.
</p>

<p>
A standard practice is to create a
branch and push it to the Randoop repository or to your own fork.  Travis will
test it; if it succeeds, you can merge it.  If it fails, you will be notified by
email, and you can fix it without having destabilized the master branch.
</p>

<p>
The Travis tests run in a Docker container.  If you are not able to reproduce a Travis test failure, then run the Randoop tests under Docker.  To do so, first start a Docker container:
</p>
<pre>
docker pull mdernst/ubuntu-for-randoop-jdk8
docker run -it mdernst/ubuntu-for-randoop-jdk8 /bin/bash
</pre>
<p>
Then, in the Docker container:
</p>
<pre>
git clone --quiet -b BRANCHNAME --depth 9 https://github.com/USER/randoop.git randoop-fork-USER-branch-BRANCHNAME
cd randoop-fork-USER-branch-BRANCHNAME
./.travis-build.sh test
</pre>


<h2 id="addtests">Adding tests</h2>

<p>
  As you add new functionality to Randoop, you should also add new tests.
  Randoop has three kinds of tests:
  tests of Randoop internal classes (under directory <code>src/test</code>),
  tests of Randoop Java agents (under directories <code>src/coveredtest</code>
  and <code>src/replacecalltest</code>), and
  system tests that run Randoop on input classes.
  All of the tests are written as JUnit classes, but are separate because each
  needs to be executed differently.
</p>


<h3 id="unit-tests">Unit tests</h3>

<p>
  If you write JUnit tests that change the static state of either a command-line
  argument or a static class, then you should add <code>BeforeClass</code> and <code>AfterClass</code>
  annotated methods that save and restore the prior state.
  For command-line arguments, this can be done by adding the following code to
  the beginning of the test class:
</p>

<pre><code>
  private static OptionsCache optionsCache;

  @BeforeClass
  public static void setup() {
    optionsCache = new OptionsCache();
    optionsCache.saveState();
  }

  @AfterClass
  public static void restore() {
    optionsCache.restoreState();
  }
</code></pre>

<p>
This uses the <code>randoop.main.OptionsCache</code> class to save the static state of
all classes that include the definition of command-line arguments.
For classes under test with static state that is changed, you would use the
<code>randoop.reflection.StaticCache</code> class to save and restore the state
of that particular class in a similar way.
</p>

<h3 id="system-tests">System tests</h3>

<p>
  The system tests are test methods of class <code>randoop.main.RandoopSystemTest</code>
  in the <code>src/systemtest</code> directory.
  Each test uses one or more input classes from the <i>testInput</i> source set,
  which resides in the <code>src/testinput/java</code> subdirectory.
  (System tests are run with a different classpath than the other tests, and
  will only take input classes from <code>src/testinput</code>.)
  So, to set up a new system test, you need to add any new input classes, and add
  a new test method to the system test class.
</p>

<p>
  Here is an example test from <code>randoop.main.RandoopSystemTest</code> that
  illustrates the main themes that occur in writing a system test.
</p>
<pre><code>
 @Test
 public void runLiteralsTest() {

   // 1. Set up a working directory -- choose a unique name
  TestEnvironment testEnvironment =
        systemTestEnvironment.createTestEnvironment("literals-test");

   // 2. Set options for test generation
  RandoopOptions options = RandoopOptions.createOptions(testEnvironment);
  options.setPackageName("");
  options.setRegressionBasename("LiteralsReg");
  options.setErrorBasename("LiteralsErr");
  options.setOption("inputlimit", "1000");
  options.addTestClass("randoop.literals.A");
  options.addTestClass("randoop.literals.A2");
  options.addTestClass("randoop.literals.B");
  options.setOption("literals-level", "CLASS");
  options.setOption("literals-file", "resources/systemTest/literalsfile.txt");

  // 3. Indicate whether you expect regression tests (if you don't care use ExpectedTests.DONT_CARE)
  ExpectedTests expectedRegressionTests = ExpectedTests.SOME;
  ExpectedTests expectedErrorTests = ExpectedTests.NONE;

  // 4. Run Randoop, compile and run generated tests, and check to see if expectations met
  generateAndTest(testEnvironment, options, expectedRegressionTests, expectedErrorTests);
 }
</code></pre>

<p>
The options list is exactly what you would give at the command line, except for
setting the names and package for generated tests.
All Randoop options may be given, except for <code>--junit-reflection-allowed=false</code>
since the test run method only looks for JUnit suite classes.
</p>

<p>
  Most existing tests take this form, using the <code>RandoopSystemTest.generateAndTest()</code> method,
  which checks whether Randoop generates the expected tests, that the tests compile,
  and that regression tests pass and error tests fail when run.
  The method also allows indicating for both kinds of tests, whether there should
  be some (at least one) test, none, or if it doesn't matter.
  This is done using the <code>ExpectedTests</code> enumerated type, which has
  values <code>SOME</code>, <code>NONE</code>, and <code>DONT_CARE</code>.
</p>
<p>
  However, it is possible to build a test differently using the utility methods of
  the <code>RandoopSystemTest</code> class.
  At a minimum, a system test always attempts to compile generated tests.
  It should also try to run any generated tests and confirm that regression tests
  pass, and that error tests fail.
  Additional information can be found in the <code>RandoopSystemTest</code> class.
</p>


<h1 id="editing-randoop">Editing Randoop</h1>

<h2 id="usinganide">Using an IDE</h2>

<p>
  The build script creates files for use by IntelliJ IDEA, NetBeans, and Eclipse.
</p>

<ul>
  <li>
  To work with <b>IntelliJ IDEA</b>, run the command

<pre>
  ./gradlew idea
</pre>

  at the command line in the Randoop directory.
  This uses the
  <a href="https://docs.gradle.org/current/userguide/idea_plugin.html">IntelliJ IDEA plugin</a>
  to create IDEA configuration files that IntelliJ uses when importing the project.
  You can then import the project as a Gradle project in IntelliJ IDEA, following
  any suggestions made on configuring Gradle support.
  See the Gradle tool window documentation at the <a href="https://www.jetbrains.com/help/idea/">IDEA Help</a> site
  on how to work with Gradle projects.
</li>

<li>
  To work with <b>NetBeans</b>, run the command
<pre>
  ./gradlew idea
</pre>
  at the command line in the Randoop directory.
  Then install the
  <a href="https://github.com/kelemen/netbeans-gradle-project">NetBeans plugin</a>
  in NetBeans.
  This plugin uses the configuration files generated for the IDEA plugin during
  the import into NetBeans.
</li>

<li>
  To work with <b>Eclipse</b>, run the command
<pre>
  ./gradlew eclipse
</pre>
  at the command line in the Randoop directory.
  This runs
  <a href="https://docs.gradle.org/current/userguide/eclipse_plugin.html">the Gradle Eclipse plugin</a>
  to configure the project for use with Eclipse.
  This will allow you to import Randoop as a project within Eclipse.
</li>
</ul>

<p>
  Other than <code>.gitignore</code>, the tool-specific settings are not part of the
  repository.
  If your tool creates new configuration files, please add them to the
  <code>.gitignore</code> file.
  If you edit <code>build.gradle</code>,
  please verify that it doesn't change behavior
  from the command line using the Gradle wrapper.
</p>


<h2 id="documentingchanges">Documenting changes</h2>

<p>
  Any user-visible changes should be documented in the changelog
  <code>src/docs/CHANGES.txt</code>.
  This text is used for documenting releases, so keeping it up-to-date and
  ready for public release saves time in the long run.
  Order the information by importance to the user.
  Be sure to describe the change in terms of how it affects the user.
  List all fixed bugs, which you can obtain by
  <a href="https://github.com/randoop/randoop/issues?q=is%3Aissue+is%3Aclosed+sort%3Aupdated-desc">querying
  GitHub</a>.  There is no need to describe each one individually unless
  users need to know about them individually.
  Optionally, mention any serious reported issues that are not fixed.
</p>


<h2 id="codeformatting">Formatting Randoop source code</h2>

<p>
  The Randoop source code is formatted in
  <a href="https://google.github.io/styleguide/javaguide.html">Google Java Format</a>
  by using the
   <a href="https://github.com/google/google-java-format">google-java-format</a>
  tool.
  The RandoopGradle build script is setup to allow this tool to be applied using
  the following commands:
  (Please read cautions below before running the tool.)
</p>
<ul>
  <li><code>./gradlew googleJavaFormat</code> (or <code>goJF</code>) to reformat
    all Java files in the repository.</li>
  <li><code>./gradlew verifyGoogleJavaFormat</code> (or <code>verGJF</code>)</li>
</ul>


<h2 id="maintaininglibraries">Updating libraries</h2>

<p>
  Periodically, follow the instructions in
  <code>lib/README</code> to update local libraries.
  (Background:  Gradle downloads the appropriate version of
  most of the libraries that Randoop uses.
  However, Randoop also uses other local libraries that are kept in the <code>lib</code>
  directory.)
</p>


<h2 id="modmanual">Modifying the manual</h2>

<p>
The documentation for Randoop is in the subdirectory <code>src/docs/manual</code>:
</p>
<ul>
<li> <code>src/docs/manual/index.html</code> has the user manual.
<li> <code>src/docs/manual/dev.html</code> is this developer manual.</li>
</ul>

<p>
The command <code>./gradlew manual</code> updates command-line argument
documentation in the user manual, and updates the table-of-contents in both
manuals (in the <code>src/docs/manual</code> directory).
</p>

<p>
The Javadoc is generated by the command <code>./gradlew javadoc</code>,
which places the files into <code>build/docs/api/</code>.
These files are published to the project website by the <code>publishSite</code>
task described below.
</p>

<p>
  Additional documentation that is not published to the project website is located
  in <code>src/docs</code>.  This includes the change log, diagrams,
  and project ideas documents.
</p>


<h2 id="github-site">Modifying the website</h2>

<p>
  The website, including the manuals, can be found in <code>src/docs</code>.
  These files, along with the Javadoc API, are moved to the
  <code>docs</code> directory by the <code>publishSite</code> task.
</p>


<h1 id="making_new_dist">Releasing a new version of Randoop</h1>

<p>
  To make a release from the master branch, follow these steps to ensure
  there are no changes needed:
</p>
<ol>
  <li>Ensure that your clone is up to date:
    <pre>git pull</pre>
  </li>
  <li>Compile, test, and build documentation.
    If there is any failure, fix them and start over.
    <pre>./gradlew clean build javadoc validateAPI manual validateSite</pre>
    This takes around 30 minutes.  To shortcut it, omit <code>clean build</code>.
  </li>
  <li>Double-check that the changelog <code>src/docs/CHANGES.txt</code>
    contains all user-visible changes in
    <pre>git log v$OLDVERSION..</pre>
    Have a colleague review the changelog entry.
  </li>
  <li>Commit any changes; push any unpushed
    commits.  Find out if any exist by running:
    <pre>git status && git log --branches --not --remotes</pre>
  </li>
</ol>
<p>
  Once there are no more changes to make:
</p>
<ol>
<li>
  Increment the Randoop version number:
  <ol>
    <li>
      Set environment variables for the old and new version numbers.  (Don't use the examples given here!)
      <pre>
OLDVERSION=<em>4.1.0</em>
NEWVERSION=<em>4.1.1</em></pre>
    </li>
    <li>Update the version number and date in these files:
<pre>
build.gradle
src/docs/CHANGES.txt
src/docs/manual/dev.html
src/docs/manual/index.html
src/main/java/randoop/Globals.java
</pre>
The following commands do part of the work.
<pre>
  preplace `echo $OLDVERSION | sed 's/\./\\\./g'` $NEWVERSION build.gradle
  cd src
  preplace `echo $OLDVERSION | sed 's/\./\\\./g'` $NEWVERSION 
  cd ..
</pre>
    You still need to manually edit <code>src/docs/CHANGES.txt</code>:
    update the date and undo undesirable changes.
    Also, double-check all changes.
    </li>
    <li>Add an exception for the Randoop zip file
      to <a href="https://github.com/plume-lib/checklink/blob/master/checklink-args.txt">checklink-args</a>,
      for instance by editing local file <code>~/bin/src/checklink/checklink-args.txt</code>.
      Don't forget to commit and push your changes.
    </li>
  </ol>
</li>
<li>Create the distribution files, commit, make a tag, and push.  This takes
  a while because it runs all the tests.  It also updates the project site in
  the <code>docs</code> directory.
<pre>
  ./gradlew clean buildRelease \
  && git add -A . \
  && git commit . -m "Update version to $NEWVERSION" \
  && git push</pre>
  This runs the tests again, so it takes about half an hour.
  </li>
  <li>
    Check that there are no broken links on the published website:
    <pre>./gradlew checklink</pre>
    This command creates file <code>checklink-log.txt</code>.
    One error is expected: <code>404</code> because file <code>randoop-$NEWVERSION.zip</code>
    does not exist yet.
        If there are any other errors, then fix them by updating URLs in
        Randoop or by suppressing a warning in
        file <code>checklink-args.txt</code>, which was mentioned above.
        If you make any changes to Randoop, then commit and push the
        changes, then start over at the <code>./gradlew clean
        buildRelease</code> step.
  </li>
  <li>
    Tag the release.
<pre>
  git tag -a v$NEWVERSION -m "Randoop version $NEWVERSION"
  git push --tags</pre>
    <!-- The gh-pages branch is not tagged. -->
    Wait for the <a href="https://travis-ci.org/randoop/randoop">Travis CI</a>
    build to succeed.  (This takes about half an hour.)
    If it fails, commit fixes, then re-tag and re-push tags
    (adding <code>-f</code> to both commands).
  </li>
  <li>Follow the
    <a href="https://help.github.com/articles/creating-releases/">GitHub instructions for creating a release</a>;
    in particular, start at
    <a href="https://github.com/randoop/randoop/releases">https://github.com/randoop/randoop/releases</a>,
    click "Tags", click "add release notes" for the new tag,
    name it "Randoop version <em>4.1.0</em>", use the changelog entry as the
    description (but remove any line breaks and check the preview), upload the
    jar files from <code>build/libs</code> and the zip file from
    <code>build/distributions</code> (it doesn't matter what order you upload
    them in; they will be listed alphabetically on the release webpage)
    and, finally, click "publish release".
  </li>
  <li>Email an announcement to randoop-discuss@googlegroups.com.
    <br/>
    Subject: Randoop version <em>4.1.0</em>
    <br/>
    Randoop <em>4.1.0</em> has been released and is available at
    https://github.com/randoop/randoop/releases/latest.  The changelog appears below.
  </li>
  <li>Make pull requests against downstream projects that use Randoop.  In each, search for the old version number.
    <ul>
      <li>
        Defects4J.  As of this writing, the Randoop version number appears only
        in files <code>init.sh</code> and <code>framework/test/README.md</code>.
      </li>
      <li>
        MUSE.  Update the <code>integration-test2</code> repository
        (the version number appears twice in the URL) and
        optionally the <code>pascali-coverage</code> repository
        (documentation only).
      </li>
    </ul>
  </li>
</ol>


<h1 id="randoop_internals">Randoop internals</h1>

<p>
This section describes Randoop's main concepts, data structures and
internal algorithms. To this end, it helps to understand our model of
unit tests.
</p>

<h2 id="unit_tests">Unit test concepts</h2>

<p>
A unit test is a snippet of code that checks
some expected property of some classes under test.
It is helpful to think of a unit test as consisting of <i>two</i> things:
</p>
<ul>
<li><b>A test input</b>:  the portions of the code that call
constructors and method of the classes under test; this is sometimes called the setup code.</li>
<li><b>Checks</b> or assertions: the
portion of the code that <i>checks</i> that the test input behaves as
expected.</li>
</ul>
<p>
As a simple example, suppose we are testing the JDK's collections
classes (<code>LinkedList</code>, <code>Collections</code>, etc.)  The
following unit test checks that the size of a linked list is computed
correctly after an element is added.
</p>

<pre class="code">
1.   // Tests that the size of a list is 1 after adding an element.
2.   public void testAdd() {
3.      LinkedList&lt;String&gt; list = new LinkedList&lt;&gt;();
4.      boolean b = list.add(null);
5.      assertTrue(list.size() == 1);
6.   }
</pre>

<p>
The test input consists of lines 3-4, which exercises the code of the list class
by creating a list, and then calling the <code>add</code> method.
The test check is on line 5.
</p>

<p>
In Randoop, a test input is represented as a
<a href="../api/randoop/sequence/Sequence.html">Sequence</a>,
and test checks are represented using <a href="../api/randoop/test/Check.html">Checks</a>.
Each sequence is made up of one or more
<a href="../api/randoop/sequence/Statement.html">Statement</a> objects.
</p>

<p>
Randoop's generation algorithms creates Sequences, adding the appropriate Checks
to each, and output the results to the user as unit tests.
</p>

<h2 id="sequence">Sequences</h2>

<p>
Randoop generates
<a href="../api/randoop/sequence/Sequence.html">Sequence</a> objects, which are used
to construct the inputs of unit tests by adding checks.
In Randoop, all test inputs are sequences of operations that use the methods and
constructors of the classes under test.
</p>

<p>
Continuing with our running example,
here is an example of a sequence of statements that calls
methods from the classes under test:
separate line:
</p>

<pre class="code">
  LinkedList&lt;String&gt; l = new LinkedList&lt;&gt;();
  String str = "hi!"
  l.addFirst(str);
  int i = l.size();
  TreeSet&lt;String&gt; t = new TreeSet&lt;&gt;(l);
  Set&lt;String&gt; s = Collections.synchronizedSet(t);
</pre>

<p>
Notice that each statement has three elements:
</p>
<ol>
<li>A specific method (or constructor) that is being called.</li>
<li>A value (primitive or object) returned by each call
(<code>l</code>,
<code>str</code>,
<code>i</code>,
<code>t</code>, and
<code>s</code>).</li>
<li>Inputs to the call, all of which come from values produced in earlier statements.</li>
</ol>
<p>
Rearranging the above sequence, we can see these elements more clearly:
</p>
<pre class="code">
                  Result      Operation        Inputs
                  ======      =============        ======
statement 0:      l       =   new LinkedList       ()
statement 1:      str     =   "hi!"                ()
statement 2:                  addFirst             (l, str)
statement 3:      i       =   size                 (l)
statement 4:      t       =   new TreeSet          (l)
statement 5:      s       =   synchronizedSet      (t)
</pre>


The three elements of a statement map to the following classes in Randoop:

<ul>

<li>
<p>A <a href="../api/randoop/operation/Operation.html">Operation</a>
represents the kind of operation that the statement
performs.
</p>
<p>
Randoop operations include:
</p>
<ul>
<li><a href="../api/randoop/operation/MethodCall.html">MethodCall</a> represents a particular
method call. This class is Randoop's analogue of Java
reflection's
<a href="https://docs.oracle.com/javase/8/docs/api/java/lang/reflect/Method.html"><code>Method</code></a>
class, and adds additional
functionality useful to Randoop.
</li>

<li><a href="../api/randoop/operation/ConstructorCall.html">ConstructorCall</a>
represents a constructor call.
</li>

<li>
<a href="../api/randoop/operation/FieldGet.html">FieldGet</a> and <a href="../api/randoop/operation/FieldSet.html">FieldSet</a>
represent getting and setting a public field. Fields are represented by
instances of classes that implement <a href="../api/randoop/field/AccessibleField.html">AccessibleField</a>.
</li>

<li>
<a href="../api/randoop/operation/EnumConstant.html">EnumConstant</a>
represents an enum constant value.
</li>

<li><a href="../api/randoop/operation/NonreceiverTerm.html">NonreceiverTerm</a>
represents a statement that declares and initializes a variable to primitive value, a String, or
the <code>null</code> value (in code, <code>int x =
1</code>, <code>String s = "s"</code>, <code>Foo f = null</code>,
etc.).
</li>

<li><a href="../api/randoop/operation/ArrayCreation.html">ArrayCreation</a>
represents a statement that declares and initializes an array out of values created in
previous statements (e.g. <code>int[] x = new int[] { var0, var1
};</code>).
</li>

</ul>
</li>

<li><a href="../api/randoop/sequence/Variable.html">Variables</a> represent
both the inputs and the outputs of a statement. A variable is just a
wrapper around an index in the sequence: given a
sequence <code>s</code>, the variable <code>s.getVariable(i)</code>
represents the value produced by the i-th statement in the sequence.
</li>

</ul>

<h3 id="creating_sequence">Creating sequences</h3>

<p>
You can create sequences in a variety of ways. The first way is
via <i>extension</i>: take a sequence and add a new statement at the
bottom. Recall the example sequence:
</p>

<pre class="code">
  LinkedList&lt;String&gt; l = new LinkedList&lt;&gt;();
  String str = "hi!";
  l.addFirst(str);
  int i = l.size();
  TreeSet&lt;String&gt; t = new TreeSet&lt;&gt;(l);
  Set s = Collections.synchronizedSet(t);
</pre>

<p>
To create this sequence by extension we need the operations.
Because this code involves parameterized types, we need to instantiate the generic
class types that are used in order to get the type substitution that we need.
For instance, for the <code>LinkedList&lt;String&gt;</code> constructor call, we
write:
</p>

<pre class="code">
  InstantiatedType linkedListType = JDKTypes.LINKED_LIST_TYPE.instantiate(ConcreteTypes.STRING_TYPE);
  Substitution&lt;ReferenceType&gt; substLL = linkedListType.getTypeSubstitution();
  TypedOperation newLL = TypedOperation.forConstructor(LinkedList.class.getConstructor()).apply(substLL);
</pre>

<p>
which first creates the parameterized type, gets the type substitution, and then
applies the substitution to the operation created from the constructor of the
generic class.
The next operations aren't from generic classes, and don't require this amount of setup:
</p>

<pre class="code">
  TypedOperation newOb = TypedOperation.createPrimitiveInitialization(ConcreteTypes.STRING_TYPE, "hi!");
  TypedOperation addFirst = TypedOperation.forMethod(LinkedList.class.getMethod("addFirst", Object.class)).apply(substLL);
  TypedOperation size = TypedOperation.forMethod(LinkedList.class.getMethod("size")).apply(substLL);
</pre>

<p>
But operations with wildcards require a substitution be applied first for the
variables in the type, a capture conversion be applied, and then a substitution
for the variables from the capture conversion.
</p>

<pre class="code">
  InstantiatedType treeSetType = JDKTypes.TREE_SET_TYPE.instantiate(ConcreteTypes.STRING_TYPE);
  Substitution&lt;ReferenceType&gt; substTS = treeSetType.getTypeSubstitution();
  TypedOperation wcTS = TypedOperation.forConstructor(TreeSet.class.getConstructor(Collection.class)).apply(substTS).applyCaptureConversion();
  Substitution&lt;ReferenceType&gt; substWC = Substitution.forArgs(wcTS.getTypeParameters(), (ReferenceType)ConcreteTypes.STRING_TYPE);
  TypedOperation newTS = wcTS.apply(substWC);
</pre>

<p>
And, similarly a generic operation requires building a substitution for the
parameter to the method:
</p>

<pre class="code">
   TypedOperation syncA = TypedOperation.forMethod(Collections.class.getMethod("synchronizedSet", Set.class));
   Substitution&lt;ReferenceType&gt; substA = Substitution.forArgs(syncA.getTypeParameters(), (ReferenceType)ConcreteTypes.STRING_TYPE);
   TypedOperation syncS = syncA.apply(substA);
</pre>

<p>
Once the operations are defined with the proper types, we can build the sequence
by extension:
</p>

<pre class="code">
  Sequence s = new Sequence();
     s = s.extend(newLL);
     s = s.extend(newOb);
     s = s.extend(addFirst, s.getVariable(0), s.getVariable(1));
     s = s.extend(size,    s.getVariable(0));
     s = s.extend(newTS,   s.getVariable(0));
     s = s.extend(syncS,   s.getVariable(4));
</pre>

<p>
A couple notable points:
</p>
<ul>
<li> Sequences are <b>immutable</b>. The <code>extend</code> operation
returns a new sequence rather than modifying its receiver.</li>
<li> If the above code feels somewhat onerous, keep in mind that the
Sequences classes were designed for automatic generation, not to
create individual sequences by hand. The purpose of these examples is
to show you how sequences are created so that, if you need to, you can
create new automated generation techniques.</li>
</ul>
<p>
A second way to create a sequence is using <code>concatenation</code>:
given sequences <code>s1</code>, <code>s2</code>, and <code>s3</code>,
you can create a new sequence that is the concatenation of the three
sequences.
</p>
<pre class="code">
List&lt;Sequence&gt; seqs = new ArrayList&lt;Sequence&gt;();
seqs.add(s1);
seqs.add(s2);
seqs.add(s3);
Sequence newSeq = Sequence.concatenate(seqs);
</pre>

<p>
If you're wondering why concatenation is useful, it is actually how
Randoop combines sequences to create new ones. To create a new
sequence that tests method <code>m(A a, B b)</code>, Randoop first
finds and concatenates previously-generated sequences that create
objects of type <code>A</code> and <code>B</code>, and then extends
the concatenated sequence with a call of <code>m</code>. See
<a href="http://homes.cs.washington.edu/~mernst/pubs/feedback-testgen-icse2007.pdf">Section
2.2. of this paper</a> for more details.
</p>
<p>
A third way to create a sequence is by parsing it from a String. For
example, given a string <code>parseable</code> with the following contents:
</p>
<pre class="code">
var0 =  cons : java.util.LinkedList.&lt;init&gt;() :
var1 =  cons : java.lang.Object.&lt;init&gt;() :
var2 =  method : java.util.LinkedList.addFirst(java.lang.Object) : var0 var1
var3 =  method : java.util.LinkedList.size() : var0
var4 =  cons : java.util.TreeSet.&lt;init&gt;(java.util.Collection) : var0
var5 =  method : java.util.Collections.synchronizedSet(java.util.Set) : var4
</pre>
<p>
The following call creates a sequence corresponding to our
running-example sequence:
</p>
<pre class="code">
Sequence seq = Sequence.parse(parseable);
</pre>

<h2 id="executing_sequence">Executable sequences</h2>

<p>
An <a href="../api/randoop/sequence/ExecutableSequence.html">ExecutableSequence</a>
 wraps a <code>Sequence</code> and adds two pieces of
 functionality:
</p>
<ul>
<li> <b>An <code>ExecutableSequence</code> can be augmented with
checks of expected properties.</b>  A <a href="#checks">Check</a> is
an object that represents some <i>expected property</i> of a sequence;
for a example, that a specific method call in the sequence returns
normally. When an executable sequence is executed, any checks that are
present in the sequence are checked at runtime, and the
passing/failing status of the checks is available for the client to
inspect.
</li>
<li> <b>An <code>ExecutableSequence</code> can be executed.</b>
Randoop uses Java's reflection mechanism to call the methods and
constructors in the sequence, and uses the structure of the
underlying <code>Sequence</code> to determine what inputs to pass to
them.
</li>
</ul>


<h3 id="executing-a-sequence">Executing a sequence</h3>

<p>
Suppose we have created a Sequence <code>s</code> The following two
lines will execute the sequence:
</p>
<pre class="code">
ExecutableSequence es = new ExecutableSequence(s);
es.execute(null);
</pre>
<p>
After the <code>execute</code> method returns, you can access the
runtime objects created during execution via the <code>getResult(int
i)</code> method, which returns the result of executing
statement <code>i</code>. For example, assuming <code>s</code> refers
to our running-example sequence from above, the following code prints
out
<code>1</code>, i.e. the return value of the call to
method <code>LinkedList.size()</code>.
</p>
<pre class="code">
  ExecutableSequence es = new ExecutableSequence(s);
  es.execute(null);

  // Assuming statement at index 3 returned normally, print the runtime value
  ExecutionOutcome resultAt3 = es.getResult(3);
  if (resultAt3 instanceof NormalExecution) {
    System.out.println(((NormalExecution)resultAt3).getRuntimeValue());
  }
</pre>
<p>
To dig more into execution results, explore the
classes <a href="../api/randoop/sequence/Execution.html">Execution</a> and
<a href="../api/randoop/ExecutionOutcome.html">ExecutionOutcome</a>.
</p>

<h3 id="miscellaneous-notes">Miscellaneous notes</h3>

<ul>
<li>If the code under test prints to standard out,
then executing statements in a sequence may result in output
to standard out. Capturing this output is possible via
the method
<code>ExecutableSequence.executeStatement</code>.  This is
accomplished by changing stdout and stderr to a memory-based
PrintSteam and recording the results.
</li>
</ul>

<h2 id="inout_sequence">Writing/reading sequences to file</h2>

<h3 id="writing_sequence_as_junit">Writing a sequence as a JUnit test</h3>
<p>
Writing a list of executed sequences as JUnit files:
</p>
<pre class="code">
  List&lt;ExecutableSequence&gt; sequences = ...;
  JunitFileWriter jfw =
  new JunitFileWriter(output_dir, junit_package_name, junit_classname, testsperfile);
  jfw.createJunitFiles(seq);
</pre>
<p>
If you want to modify Randoop's JUnit-generating code, here are places to look:
</p>
<ul>
<li>Class <code>randoop.JunitFileWriter</code> figures out how many JUnit classes/files
to write, how many tests to put in each, what to name them, etc.
  <ul>
  <li><code>randoop.JunitFileWriter.writeDriverFile</code> generates the non-reflective driver file</li>
  <li><code>randoop.JunitFileWriter.writeSuiteFile</code> generates the (reflective) test suite</li>
  </ul>
</li>
<li>Methods responsible for writing the code for a single unit test:
  <ul>
  <li><code>randoop.ExecutableSequence.toCodeString()</code>,</li>
  <li>implementors of <code>randoop.ObjectContract</code>,</li>
  <li>implementors of <code>randoop.Check</code>,</li>
  <li>and the various implementations of <code>Operation.appendCode(...)</code></li>
  </ul>
</li>
</ul>


<h2 id="checks">Checks</h2>

<p>
A <a href="../api/randoop/test/Check.html">Check</a> is an object that
represents an <b>expected property</b> of a sequence. In terms of
 <a href="#unit_tests">unit test concepts</a>, a check represents some
piece of checking code (or &ldquo;test oracle&rdquo;) of a unit test.
</p>
<p>
For example, consider the following unit test:
</p>
<pre class="code">
 1.  // Tests that the size of a list is 1 after adding an element.
 2.  @Test
 3.  public void testAdd() {
 4.     LinkedList l = new LinkedList();            // Create a list.
 5.     assertTrue(l.equals(l));                    // List should be equal to itself.
 6.     Object o = new Object();
 7.     boolean b = l.add(o);                       // Add an element to it.
 8.     org.junit.Assert.assertTrue(l.size() == 1); // List should have size 1.
 9.     org.junit.Assert.assertTrue(l.equals(l));   // List should still be equal to itself.
10.     int i = 10;
11.     try {
12.       Object o2 = l.remove(i);                  // Removing from invalid index
13.       org.junit.Assert.fail();                  // should throw exception.
14.     } catch (IndexOutOfBoundsException e) {
15.       // expected exception.
16.     }
17.  }
</pre>
<p>
Like a typical unit test, this test combines <i>test input code</i>
(the code creating a list, adding an element to it, removing an
element, etc.)  with <i>checking code</i> (the code in lines 6-7
creating assertions, and the code in lines 9, 11-14 checking that the
correct exception is thrown). In Randoop, test input code is
represented as a <a href="#sequence">Sequence</a> and checking code is
represented by <code>Check</code>s associated with the sequence.
</p>
<p>
More specifically, each check is associated with an index in the
sequence, where the index represents the time at which the check
should be performed. For example, rearranging the above test to better
see its structure, we have:
</p>

<pre class="code">

                          Checks performed
Statement                 after statement
=========                 ===============
l  = new LinkedList()
                           l.equals(l) returns true.
o  = new Object()
                           no checks.
b  = l.add(o)
                           l.size() returns 1.
                           l.equals(l) returns true.
i = 10;
                           no checks.
o2 = l.remove(i)
                           throws IndexOutOfBoundsException.
</pre>
<p>
The example illustrates how Randoop represents a test input along with
its correctness checks.
</p>

<h3 id="distinction">Statements vs. checks</h3>
<p>
Looking at the above example, you may reasonably ask: why do we draw a
distinction between &ldquo;test input code&rdquo; (the statements on the left) and
&ldquo;checking code&rdquo; (the statements on the right)? After all, aren't the
calls to <code>size</code> and <code>equals</code> in the checking
column also calls of the classes under test, and why not consider them
part of the test input?
</p>
<p>
We offer an answer in terms of how Randoop works. Randoop generates
test inputs (<a href="#sequence">Sequences</a>) <i> randomly</i>, by
combining and extending previously-generated sequences. On the other
hand, Randoop performs its checks <i>deterministically</i>. For every
sequence it creates, it performs checks on all the objects of the
sequence.
</p>
<p>
Given how Randoop treats sequences and checks differently, it makes
more sense for a call like <code>l.equals(l)</code> to be expressed as
a <a href="../api/randoop/test/Check.html">Check</a>, so that it is always
(not randomly) performed.
</p>

<h3 id="checks_">Executing Checks</h3>
<p>
(Under construction)
</p>

<h2 id="code-entry">Main entry points</h2>

<ul>
<li><a href="../api/randoop/main/GenTests.html">randoop.main.GenTests</a>
is the main class for Randoop as it is normally used.  There are other
mains for other purposes.
</li>
<li>Method <code>handle</code> is the
main <a href="../api/randoop/main/GenTests.html">GenTests</a>
entrypoint for Randoop. (This is not strictly true, as Randoop's true
entrypoint is
class <a href="../api/randoop/main/Main.html">randoop.main.Main</a>.
But <code>GenTests</code> is where all the action starts with test
generation.) The <code>handle</code> method is long and mostly deals with
setting up things before the generation process, and doing things like
outputting tests after generation.
</li>
<li>Most command line options are specified
in <a href="../api/randoop/main/GenInputsAbstract.html">GenInputsAbstract</a>.
</li>
</ul>


<h2 id="code-generator">Test generator classes</h2>

<ul>
<li><code>ForwardGenerator</code> is the generator for Randoop's normal operation.</li>
</ul>

</body>
</html>

<!--  LocalWords:  Randoop Randoop's Makefile classpath bashrc hoc gentests JDK java gh EnumConstant enum s1 s2 s3 var0 PrintSteam plugin agentTest gradle taskName RandoopSystemTest plugins gitignore randoop changelog RandoopGradle goJF verGJF cd OLDVERSION NEWVERSION unpushed mv subprojects src agenttest  testinput systemtest FieldGetter FieldSetter PublicField init var1 var2 var3 var4 var5 resultAt3 assertTrue o2 testInput str FieldGet FieldSet AccessibleField NonreceiverTerm ArrayCreation InstantiatedType linkedListType ConcreteTypes ReferenceType substLL getTypeSubstitution TypedOperation forConstructor createPrimitiveInitialization forMethod treeSetType substTS gradlew subproject validateManual TODO checklink txt boolean wildcards wcTS substWC syncA substA implementors IndexOutOfBoundsException deterministically sudo qqy jdk perl html5validator sourceset cp sourcepath ErrorTest suitepath ErrorTest0 suiteclasspath Defects4j md Pascali mdernst ubuntu jdk8 BRANCHNAME javadoc validateAPI valdateSite preplace buildRelease applyCaptureConversion forArgs getTypeParameters
 -->
<!--  LocalWords:  google JDK's testAdd LinkedList addFirst TreeSet MethodCall th
 -->
<!--  LocalWords:  synchronizedSet Operation ConstructorCall ArrayDeclaration
 -->
<!--  LocalWords:  PrimitiveOrStringOrNullDecl newLL newOb newTS
 -->
<!--  LocalWords:  getRConstructor addFist getRMethod syncS getVariable seqs gz
 -->
<!--  LocalWords:  ArrayList newSeq parseable ExecutableSequence runtime stdout
 -->
<!--  LocalWords:  ExecutionOutcome resultAt getResult instanceof stderr fileos
 -->
<!--  LocalWords:  NormalExecution getRuntimeValue printsteam FileOutputStream
 -->
<!--  LocalWords:  MyFile ObjectOutputStream objectos GZIPOutputStream fileis
 -->
<!--  LocalWords:  writeObject FileInputStream ObjectInputStream objectis JUnit
 -->
<!--  LocalWords:  GZIPInputStream seqsfromfile readObject deserialized jfw dir
 -->
<!--  LocalWords:  JunitFileWriter junit classname testsperfile
 -->
<!--  LocalWords:  createJunitFiles GenTests entrypoint GenInputsAbstract
 -->
<!--  LocalWords:  ForwardGenerator
 -->
